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Multispectral imaging is a highly desirable modality for material-based analysis in diverse areas such as food 
production and processing, satellite-based reconnaissance, and biomedical imaging. Here, we present 
nanofilter-based multispectral videography (nMSV) in the 700 to 950 nm range made possible by the tunable 
extraordinary-optical-transmission properties of 3D metallic nanostructures. Measurements made with nMSV 
during a bolus injection of an intravascular tracer in the ear of a piglet resulted in spectral videos of the 
microvasculature. Analysis of the multispectral videos generated contrast measurements representative of 
arterial pulsation, the distribution of microvascular transit times, as well as a separation of the venous and 
arterial signals arising from within the tissue. Therefore, nMSV is capable of acquiring serial multispectral 
images relevant to tissue hemodynamics, which may have application to the detection and identification of skin 



The interaction of light with surface plasmons (SPs) at an interface between a metal and a dielectric has enabled 
applications ranging from molecular sensing to cancer therapy 1 . A metallic structure with sub-wavelength 
features (e.g. a periodic array of nanoholes) allows light-SP coupling and produces unexpected characteristics 
such as transmission, reflection, and absorption resonances 2 . Nanohole array (NHA) structures have been shown 
capable of extraordinary optical transmission (EOT), which has enabled new optical techniques such as sub- 
diffraction light focusing (super-lensing) and improved upon existing techniques such as particle trapping and 
Raman spectroscopy 16 . The dependence of the EOT on material composition and the geometrical parameters of 
NHAs has been studied by several groups 79 . For example, the transmission resonance position of a NHA can be 
tuned in the visible and near infrared regions by changing the spacing between the holes or selecting the refractive 
index of dielectric material on either side of the perforated metallic film 7,8 . Moreover, transmission efficiency and 
bandwidth of the EOT can be refined via nanohole size and arrangement 8,10 . The high degree of tunability coupled 
with device compactness have motivated several groups to test the usefulness of metallic NHA structures as optical 
elements for cameras, displays, and photography 1114 . For example, NHA structures have been used to create color 
photos with feature sizes below the optical diffraction limit, a feat not feasible with other optical techniques 12 . 
Furthermore, recent developments in the fabrication of NHAs on complementary metal-oxide semiconductor 
(CMOS) sensors have been tested with the goal of replacing dye-based RGB filters 11,14 " 16 . However, a demonstra- 
tion of color imaging with NHA-CMOS devices is still lacking, likely due to wide transmission resonances and 
poor out-of-band blocking for NHAs fabricated on top of the CMOS sensor. Our group has recently reported on 
NHA structures with greatly improved EOT properties that enabled snapshot multispectral imaging (MSI) in both 
the visible and NIR spectral ranges. These enhanced NHA structures incorporated a cavity beneath each nanohole 
that resulted in SP-energy-matching of the EOTs at the top and bottom surfaces of the metal film. Compared to 
conventional NHA structures, the SP-energy-matched NHA structures demonstrated superior EOT character- 
istics such as deeper out-of-band blocking, narrower resonance bandwidth, and higher transmission efficiency 17,18 . 

Multispectral videography (MSV) has long been desirable for the capture of spectral information from moving 
objects and flow in various fields such as remote-sensing and biomedical imaging, but true MSV (i.e. simultan- 
eous 2D image capture of many spectral bands) has been technically challenging due to the spatial or spectral 
scanning processes inherent to multispectral imaging systems 19 " 21 . Scanning can result in images with motion 
artefacts such as blur 19 . Recent developments in patterned filters have enabled MSI with camera sensors. For 
example, dichroic filters have been micro-patterned in a color filter array arrangement and shown capable of 
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multispectral imaging in four spectral bands within the NIR region 22 . 
The fabrication difficulty of patterned dichroic filters has been a 
limitation of this technology 23 . With the goal of developing MSI 
devices with compactness, easy fabrication, and higher image reso- 
lution, plasmonic nanostructures have been recently implemented 
for MSI imaging within visible and NIR spectral regions 23-24 . As an 
alternative to other optical filters, our group reported on a MSI 
system based on a tiled-mosaic arrangement of SP-energy-matched 
NHAs. This system was shown capable of snapshot multispectral 
imaging in both transmission and reflection modes for detection 
bands in the visible and NIR optical regions 24 . 

In this paper, we describe nanofilter-based MSV (nMSV) using a 
tiled-mosaic arrangement of SP-energy-matched NHAs to capture 
videos representative of 4 optical bands in the NIR spectral region. 
The apparatus included a second image sensor that simultaneously 
captured color images of a scene that were co-registered to the multi- 
spectral images. The apparatus proved capable of MSV for analysis of 
fast processes such as the dispersion of droplets into water in an open 
cuvette, the flow of indocyanine green (ICG, a common optical con- 
trast agent used in biomedical imaging) through a capillary tube on a 
microscope stage, and the transit of ICG through the vasculature of 
the ear during a bolus injection in an anesthetized piglet. Further- 
more, data-reduction techniques were applied to the time-dependent 
image data, which resulted in estimates of the heart rate, the distri- 
bution of transit times through the microvasculature, and positive 
identification of near surface venous vessels and arterial components. 

Results 

To construct the nMSV prototype, we first designed and fabricated a 
NHA-based device that enabled simultaneous multispectral imaging 
in four spectral bands between 700 nm and 950 nm. Figure 1 (a) 



displays a schematic of the device consisting of 60 by 60 blocks in a 
tiled arrangement, where each block contained 4 SP-energy-matched 
NHAs with 380 nm, 425 nm, 460 nm, and 510 nm periodicities in a 
mosaic arrangement. Each NHA was 20 urn by 20 um in size, and 
spacing between adjacent NHAs was 10 um. Our previous work on 
cross-talk between closely packed NHAs suggested that a 10 urn 
spacing was adequate to reduce SP-based interaction between 
NHAs 25 . NHA structures with SP-energy-matched properties were 
used due to the higher spectral-band-pass filter performance at the 
resonance wavelengths and Wood's anomaly compared to non-SP- 
energy-matched NHA structures 7,17 . The SP-energy-matched NHA 
structures were fabricated with electron beam lithography (EBL) in a 
70-nm optically thick gold film on a 1-inch BK7 substrate. A Ti wet- 
etching process was applied to remove the Ti adhesion layer within 
each nanohole as well as to create a cavity within the BK7 substrate 
beneath each nanohole 18 . The NHA structures were spin-coated with 
a 200-nm thick layer of polymethyl methacrylate (PMMA 950 A2, 
MicroChem, Newton, MA) that filled the cavities and covered the top 
surface of the perforated gold film. The PMMA provided a protection 
layer and a common index of refraction (n=1.49 at 589 nm wave- 
length) above and below the gold film that resulted in a SP-energy- 
matching effect that was highly controlled. The ratio of the aggregate 
hole area to the film area for each NHA was 0.09 and provided similar 
(1,0) resonance transmission efficiency between NHAs within each 
block. The NHA geometrical parameters were selected in such a way 
that the (1,0) SP-energy-matched resonance peak occurred at 731 
nm, 793 nm, 855 nm, and 917 nm for periodicities of 380 nm, 425 
nm, 460 nm, and 510 nm, respectively. Figure 1 (b) displays the 
optical transmission of NHAs for 4 spectral illumination bands 
within 700 nm and 948 nm and a pass band bandwidth (FWHM) 
of 48 nm. The (1, 0) resonance and (1, 0) minimum, and the (1, 0) 
resonance SP lifetime of the NHA structure were responsible for the 




731 793 855 917 
band center wavelength (nm) 



(b) (c) 

Figure 1 | (a) Schematic diagram of color filter arrays consisting of 60 by 60 blocks (left). Each block had 4 different NHAs with a periodicity of 380 nm, 
425 nm, 460 nm, or 510 nm (middle). A SEM image showing a region of a NHA with 380 nm periodicity (right), (b) Average optical transmission of 
NHAs when independently illuminated with narrow band (FWFM = 62 nm) light at 731 nm, 793 nm, 855 nm, and 917 nm. c) Map of correlation 
coefficients between optical transmission measurements for data in panel (b). 
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improved transmission and out-of-band blocking characteristics 9 . 
The average standard deviation of the transmission for 3600 NHAs 
with the same periodicity was 3% at each spectral band, which 
demonstrated good consistency among transmission spectra of all 
NHAs. The correlation coefficient map representative of signal over- 
lap between pairs of NHAs is shown in Figure 1 (c). Most pairs of 
NHAs produced negative correlation coefficient values, which pre- 
dicted that the design parameters of the NHAs would, in principle, 
result in the separation of optical signals into 4 spectral bands 
between 700 nm and 950 nm, with minimal cross-contamination. 
Spectral separation of the incoming light into the its component 
bands was performed with a linear unmixing algorithm based on 
the non-negative least-squares estimation (LSE) technique 24,26 . In 
the LSE technique, the unmixed signal at each spectral band (x (1 
through 4)) was estimated using (jnin\\y — Ax\\ 2 ^, where y, x, A and 
|| . || 2 represent a mixed spectrum (light transmission intensity of each 
NHA (1 through 4)), the estimated abundance of the endmembers 
(estimated light intensity at each spectral band ( 1 through 4) ) , the 2D 
calibration matrix of endmember spectra, and the norm operation, 
respectively 26 . During the unmixing process, the abundance of end- 
members was constrained by x>0. The device was calibrated by 
imaging the optical transmission through the entire device for each 
illumination band independently. The 2D calibration matrix was 
constructed from 4X4 elements, where each element was the aver- 
age transmission measured for NHAs of the same periodicity and 
calibration illumination band. Column elements of A were arranged 
by NHA periodicity and row elements were arranged by center wave- 
length of the illumination band used for calibration (please refer to 
Figure 1 (b)). 

Based on the NHA device, we constructed an apparatus for per- 
forming nMSV at 4 spectral bands between 700 nm and 950 nm. 
Figure 2 (a) illustrates the apparatus, which was comprised of the 
NHA device, 2 cameras, a dichroic mirror, 2 folding mirrors, block- 
ing filters, and a series of relay lenses. The first relay lens, LI (//. = 
100 mm; broadband) and L2 (f.l. = 50 mm; AC254-50-A; Thorlabs), 
transferred a visible color image from the input focal plane of the 
apparatus onto a color CMOS sensor (Logitech Webcam Pro 9000, 
Apples, Switzerland). The input focal plane of the apparatus was 
coincident with the image plane of the input objective (i.e. telecentric 
lens). The second relay lens, LI and L3 (f.l. = 50 mm; AC254-50-B; 
Thorlabs), relayed a NIR image of the input image plane to the NHA 
device. A dichroic beam splitter DM (FF705-DiO 1-25x36, Semrock 
Inc., New York, USA) was placed between LI and the other two 
lenses (L2 and L3) to reflect the visible components toward the 
CMOS color camera, as well as to transmit the NIR light toward 
the NHA device. An image of the NHA device was collected by the 
third pair of relay lenses, L4 (f.l. = 50 mm; AC254-50-B; Thorlabs) 
and L5 (f.l. = 100 mm; AC254-100-B; Thorlabs), which then was 
focused onto the monochromatic CMOS sensor (MSV1-D1312 (1/ 
IE)-GB; 8 bit dynamic range; Photon Focus, Lachen, Switzerland). A 
950-nm short-pass and a 700-nm long-pass interference filter (Fl: 
FEL 700 and F2: FES 950, Thorlabs) were located between L4 and L5 
to pass light within the 700 nm to 950 nm band to the monochro- 
matic CMOS sensor. Two gold mirrors Ml and M2 (PF10-03-M01, 
Thorlabs) were used to fold the beam to reduce the length of the 
apparatus. 

In the first study, trans-illumination mode nMSV performance 
was tested by dye dispersal measurements. In the experimental setup, 
collimated white light from a 100 W halogen lamp was transferred 
by a liquid light guide to a beam expander to illuminate a 10 mm 
optical path quartz cuvette (3ES10, NSC Precision Cells Inc., NY, 
USA) filled with 3 mL of distilled- deionized water (DI-H 2 0). A C- 
mount telecentric lens (IX magnification; 55350, Edmond Optics, 
NJ, USA) was mounted on the apparatus to image the dispersion of 
drops of dye as they were released into the DI-H 2 0. Figure 2 (c) 
displays 3 sets of 5 frames extracted from multispectral videos of 9 



drops of 1 M copper sulfite, 5 drops of 1 mM Methylene Blue (MB + ), 
and 8 drops of 100 uM indocyanine green (ICG). To verify multi- 
spectral results, absorption spectra of 500 mM copper sulfite, 80 uM 
MB + , and 40 uM ICG were acquired with a spectrophotometer (DU 
640, Beckman Coulter, Inc., Brea, CA, USA) and shown in Figure 2 
(b). The absorption spectra for each dye was captured with a quartz 
cuvette with a 4-mm optical path and blanked to DI-H 2 0. Each drop 
had a volume of approximately 1 5 uL. Each frame consisted of a RGB 
color image, an image of the NHA device, and 4 unmixed images of 
the NHA device representative of four spectral bands between 
700 nm and 948 nm. The multispectral images were normalized to 
images of DI-H 2 0 lacking dye to obtain estimates of maximal optical 
transmission. Multispectral videos of 1M copper sulfite drops 
revealed rapid dispersal into the DI-H 2 0 and homogeneous absorp- 
tion across all 4 bands within 67.5 s of the last drop (Figure 2 (c); left). 
The last multispectral frame contained transmission contrast due to 
copper sulfite even though there was no significant contrast observ- 
able in the corresponding color camera frame. However, due to the 
difference in refractive index between copper sulfite (n = 1.514) and 
DI-H 2 0 (n=1.33), dispersion of copper sulfite drops into the DI- 
H 2 0 were apparent in the intermediate color frames. In the case of 
1 mM MB + (Figure 2 (c); middle), drops of MB + tended to disperse 
slowly and were most evident in the frames representative of the 700- 
762 nm band. Contrast due to MB + was also observed in the corres- 
ponding color camera frames. The results were expected since MB + 
has a broad absorption peak at 660 nm (within range of the color 
sensor), which extends into the 700-762 nm and 762-824 nm detec- 
tion bands (within range of the NHA device) as shown in Figure 2 
(b). Small dye-dependent signals were found in the 886-948 nm 
bands, likely due to the high concentration of MB + in the cuvette. 
At high concentrations, optical absorption was no longer linearly 
related to concentration and gave rise to an analysis artefact that 
appeared as signal contamination across unmixed detection bands 27 . 
In Figure 2 (c) (right), frames representative of drops of 100 uM ICG 
are shown as they disperse into the DI-H 2 0. Transmission contrast 
indicative of the dispersal process was observed in the images rep- 
resentative of the 700-762 nm and 762-824 nm unmixed transmis- 
sion bands. Optical transmission appeared to be highest in the frames 
representative of the 762-824 nm band and lower in the 700- 
762 nm band; this was consistent with the absorption properties of 
ICG at concentrations less than 50 uM, which has a primary absorp- 
tion peak at 780 nm and a smaller secondary absorption peak at 
705 nm. Similar to the MB + multispectral videos, a small degree of 
optical transmission contrast was observed for bands outside of the 
primary absorption bands of ICG at high concentrations. It is antici- 
pated that the leakage artefact common to regions of high concen- 
tration can be reduced with a camera of higher dynamic range 
capable of measuring the small changes in optical transmission for 
incremental changes in dye concentration when the concentration is 
high. 

The nMSV apparatus was capable of resolving optical absorption 
of various dyes into four spectral bands in the NIR simultaneously. 
The nMSV frames were of sufficiently high resolution to track the 
absorption features related to the convective movement of dye as it 
was dispersed into the DI-H 2 0. The nMSV frames were qualitatively 
similar to the corresponding higher resolution frames acquired with 
the color camera. The nMSV apparatus was capable of both tracking 
dye as it diffused into the DI-H 2 0 and clearly separating absorption 
due to dye into the expected spectral bands. Unlike other MSI sys- 
tems based on spatial and spectral scanning processes, nMSV with 
patterned 3D nanostructures was insensitive to smearing and motion 
artefacts due to the simultaneous capture of the 2D video frames at 
each spectral band. 

In a second study, the nMSV apparatus was attached to an inverted 
microscope (TE 300, Nikon, Tokyo, Japan) through a C-mount to F- 
mount adapter, which enabled MSV of objects located within the 
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Figure 2 | a) Multispectral videography apparatus consisting of achromatic visible and NIR lenses (LI, L2, L3, L4, and L5), a dichroic beam splitter 
(DM: reflective < 700 nm), gold mirrors (Ml and M2), a 950 nm short-pass interference filter (Fl), a 700 nm long-pass interference filter (F2), a color 
camera (CI), a monochrome camera (C2), and a tiled-mosaic arrangement of NHA arrays (NHA device). The illumination consisted of light transferred 
from a halogen lamp by a liquid light guide to a beam expander. The imaging target was a 10-mm optical path quartz cuvette containing 3 mL DI-H 2 0. 
Drops of dye (—15 pL per drop) were released into the DI-H 2 0 and the dispersal process was imaged, b) Spectral absorption of 500 mM copper sulfite 
and 100 pM and 200 jiM methylene blue (top panel); spectral absorption of ICG (20 pM, 50 pM, and 100 uM; bottom panel), c) Select frames of a 
multispectral video taken from three experiments. Each column of frames represents a single exposure at a specific time point (lower scale) and includes 
the color camera image (Color), the image of the NHA device (NIR), and unmixed images for each detection band (700-762 nm, 762-824 nm, 
824-886 nm, and 886-948 nm). Experiments were performed with 1 M copper sulfite drops (left), 1 mM MB + drops (middle), and 100 pM ICG drops 
(right) (See the supplementary videos for 1 M copper sulfite, 1 mM methylene blue, and 100 pM ICG drops). 



focal plane of the microscope objective (Figure 3 (a)). The setup was 
used to quantify the dynamic range of the NHA device with respect 
to dye concentration in a capillary tube (10 uL; VWR Scientific, PA, 
USA). A solution of DI-H 2 0 and ICG was pumped through the 



capillary tube to dynamically change the ICG concentration in the 
range of 0 to 1000 uM. Multispectral videos of the flow were cap- 
tured at 8 frames per second. To facilitate dynamic changes in ICG 
concentration, the capillary tube was attached to two independent 
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syringe pumps (22, Harvard Apparatus, Inc., MA, USA) with one 
pump holding a syringe loaded with DI-H 2 0 and another pump 
holding a syringe loaded with 1000 uM ICG (in DI-H 2 0). The total 
flow rate for both pumps was maintained at 50 uL/min throughout 
each experiment. Initially, the flow of DI-H 2 0 was set to 50 uL/min 
with zero flow rate on the ICG pump. Every 100 s, the flow rate on 
the ICG pump was increased by 10 uL/min and the flow rate on the 
DI-H 2 0 pump was reduced by 10 uL/min simultaneously. The step- 
ping of the flow continued until the complete multispectral video of 
the capillary tube for the full range of ICG concentrations (i.e. 0- 
1000 uM) was obtained. Figure 3 (b) displays multispectral video 



frames taken at select time points representative of each ICG con- 
centration. Consistent with the ICG absorption spectrum, unmixed 
images revealed optical absorption within the 700-762 nm and 765- 
824 nm bands, but not in the longer wavelength bands. Furthermore, 
as expected, transmission in each of the first two bands declined as 
the concentration of ICG increased. Optical transmission differed 
across the diameter of the capillary tube, which was consistent with 
the range of optical path lengths related to the circular cross-section 
of the capillary tube and refraction effects due to the curved liquid- 
glass-air interfaces. For example, optical transmission measured near 
the wall of the capillary tube (short optical path length) was higher 
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Figure 3 | (a) Multispectral videography apparatus attached to an inverted microscope using C-mount to F-mount adapter. Multispectral videos were 
collected of a capillary tube on the microscope stage, which was filled with a mixture of DI-H 2 0 and ICG. The concentration of ICG within the capillary 
tube was controlled by the ratio of the flow rate of 1000 uM ICG to the flow rate of DI-H 2 0 through two independent syringe pumps (not shown) 
connected to the capillary tube through inlets. Mixing occurred at the inlet of the capillary tube. The illumination source was the halogen lamp onboard 
the microscope. White light was focused on to the capillary tube using the condenser of the microscope, (b) Select frames of a multispectral video showing 
images representative of the color camera, the NHA device, and 4 unmixed detection bands (700-762 nm, 762-824 nm, 824-886 nm, and 886-948 nm) 
for specific ICG concentrations (0 uM, 200 uM,400 uM,600 uM, 800 uM, and 1000 uM) (See the supplementary video for various ICG concentrations 
flowing through the capillary tube), (c) Absorbance for a region of interest (represented for one band by red dashed box in panel (b)) for each detection 
band as a function of time, d) Absorbance (±1 SD) for a region of interest (red dashed box in panel (b)) as a function of ICG concentration (0 uM, 
200 uM, 400 uM, 600 uM, 800 uM, and 1000 uM) measured at time points indicated by the vertical green dashed lines in panel (c). 
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than the optical transmission measured near the center of the capil- 
lary tube (longer optical path length) for a given ICG concentration. 
A region of interest analysis of the multispectral video is shown in 
Figure 3 (c) where the absorbance for each of the 4 bands is plotted 
against time. The absorbance for a region of interest was calculated 
based on a logarithmic ratio between the average pixel intensity 
within each region of each video frame over the similarly acquired 
average pixel intensity capture with water alone. The plot clearly 
shows the increase in absorbance for the first 2 bands (700- 
762 nm and 765-824 nm) and almost no change for the last 2 bands. 
Based on the signal changes, nMSV was able to capture the mixing 
kinetics as a change in ICG concentration that approached a new 
asymptotic value after the flow rate on each pump was adjusted. The 
lack of optical transmission contrast within the last two detection 
bands was expected due to the lack of ICG absorption in these bands. 
Based on the time series data in Figure 3 (c), the dependence of 
absorbance on ICG concentration was analyzed (Figure 3 (d)). 
Absorbance in each of the first two bands increased linearly as 
ICG concentration increased. The standard deviation of the absor- 
bance measured for the first two bands ranged from ± 0.046 to ± 
1.148 for the lowest to highest ICG concentrations, respectively. The 
range in measured error was related to the range of absorbance 
measurements across the diameter of the capillary tube due to optical 
path length variations and refraction effects (see above). However, 
the standard deviation was lower for the last two bands (±0.012 and 
±0.022, respectively) due to a lack of ICG absorption. 

In a third study, nMSV was used during an animal imaging 
experiment to capture the transit of a bolus injection of ICG 
through the microvasculature of a new born Duroc piglet (< 
48 h old). The piglet was anesthesized (induction with 3-4% iso- 
flurane), tracheotomized, and mechanically ventilated during the 
experiment. A catheter was inserted into the left ear vein to facil- 
itate ICG injection (1 mL, 2 mM). A cannula was inserted into a 
femoral artery for continuous monitoring of blood pressure. The 
arterial ICG concentration was measured with a pulse dye dens- 
itometer (PDD) (DDG 2001 A/K, Nihon Kohden, Tokyo, Japan) 
attached to a hind foot. During the imaging experiment, the aver- 
age blood pressure, heart rate, respiratory rate, and body temper- 
ature of the piglet were 55 mmHg, 165 bpm, 55 breaths per 
minute, and 39.3°C, respectively. All animal experiments were 
approved by the Animal Use Subcommittee of the Canadian 
Council on Animal Care at the University of Western Ontario 
(protocol # 2011-063). 

A multispectral video of the top surface of the right ear of the piglet 
was captured at 11.54 fps. The underside of the ear opposite the 
imaging field was illuminated with a 100 W halogen lamp. The 
images were collected with a telecentric lens (IX magnification; 
55350, Edmond Optics, NJ, USA) attached to the multispectral 
videography apparatus. A vessel with a nominal diameter of 
200 um was visible across the field of view. After the ICG bolus into 
the left ear, the bolus travelled through the systemic circulation prior 
to reaching the microvasculature of the right ear. In this senario, the 
multispectral video potentially contained contrast representative of 
variations in ICG signal due to the heart, analogous to pulse dye 
densitometry. Figure 4 (a) displays selected frames of a multispectral 
video of the ear before, during and after the bolus injection of ICG. 
Inspection of the multispectral video frames relative to the start of the 
experiment revealed that ICG contrast (700-762 nm and 765- 
824 nm) appeared within 8 ± 2 s and reached a maximum value 
at the 13 ± 2 s. For superficial microvasculature (i.e. visible in the 
color camera images), the appearance and decay of ICG contrast due 
to the bolus was clearly visible and appeared coregistered with the 
superficial microvasculature morphology. Also, the overall bright- 
ness in the first two bands decreased due to the transit of ICG 
through deeper microvasculature, which were not spatially resolv- 
able due to the scattering properties of tissue. In the last two bands 



(824-886 nm and 886-948 nm), no presence of ICG contrast was 
observed; however, near surface microvasculature were observed 
with low contrast (see Figure 4 (a)). 

A region of interest analysis repeated on consecutive frames of the 
images representative of the second detection band are shown along- 
side the corresponding PDD readings (Figure 4 (b)). The ICG esti- 
mates obtained from the PDD device revealed an earlier arrival of 
ICG contrast (—4.2 seconds lag) compared to the arrival of ICG in 
the vasculature on the ear. This difference can be explained by the 
shorter arrival time of ICG to the arterioles of the hind foot where the 
PDD probe was placed, compared to the arrival of the ICG bolus to 
the venous vasculature in the ear. The signal from the second band 
representative of ICG contrast was temporally wider than the PDD 
signal due to the dispersion of the contrast agent as it passed through 
the capillary bed of the ear 28 . 

Low amplitude oscillations were apparent in the ICG time course 
and investigated using the second band images from the nMSV. The 
magnitude Fourier transform of ICG time course is shown in 
Figure 4 (c). Frequencies below 1 Hz were present and likely related 
to respiratory motion. Also, two distinct peaks were observed at 
2.75 Hz and 5.5 Hz, which corresponded to the heart rate (measured 
independently at 165 bpm) and the second harmonic of the heart 
rate. The appearance of frequencies representative of the heart rate 
can only be explained by sensitivity of the nMSV to changes in optical 
pathlength caused by the pulsation of arterial blood. 

Using the multispectral video, the transmitted light intensity at the 
ICG peak and the time to ICG peak were computed for each pixel in 
the 700-762 nm and 762-824 nm spectral band images; this is 
shown in Figures 4 (d) and 4 (e), respectively. In Figure 4(d) , the 
signal intensity at each pixel was lower for the map represenative of 
the second band compared to the map of the first band due to higher 
absorption of ICG dye in the second band. The surface-weighted 
patterns of microvasculature were observed in both maps as features 
with lower signal intensity compared to surrounding regions. 
However, other features were present in the images and were likely 
representative of the surface irregularities of the skin. The maps of 
time to ICG peak clearly revealed the surface-weighted microvascu- 
lature; however, the extent of the visualized microvasculature was 
greater in the second band due to the higher ICG absorption and 
greater signal contrast (Figure 4 (e)). Based on the time to peak in the 
second band, the ICG contrast signal appeared later in time for the 
regions representative of microvasculature compared to surrounding 
regions, which suggested that the microvasculature features are on 
the venous side of the circulation. Furthermore, one can infer that the 
darker areas are representative of the arterial side of the circulation 
since the correlation lag appeared earlier in time. However, due to the 
scattering properties of tissue and the likelihood that arteries and 
arterioles are located deeper than veins, morphological features typ- 
ical of vessels, other than the surface-weighted venous vessels, were 
not obvious in the images. Although both first and second band maps 
were similar in terms of the distribution of time to peak values, the 
second band image revealed a later time to peak compared to the first 
band image. The later arrival can be understood by interpreting the 
results within the context of the cumulative effects of the wavelength- 
dependent scatter in tissue and the absorption of ICG. Compared to 
the second spectral band, the frames related to the first spectral band 
were expected to have lower optical absorption due to ICG, but 
higher optical scattering. Therefore, in transillumination, photons 
detected in the first band were more likley to have propagated along 
longer paths compared to photons detected in the second band, and 
hence represent optical properties for deeper tissues where the arter- 
ial vasculature is denser and ICG appears early. In contrast, the 
signals present in the second band are likely to be representative of 
the superficial venous vasulature where ICG is expected to arrive 
later. This effect is consistent with earlier work (e.g. see Ref. 29) that 
demonstrated a dependence of the arrival time of ICG on detection 
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Figure 4 | Multispectral videography of the microvasculature of the outer surface of a piglet ear during a bolus injection of ICG into a vein in the 
opposite ear. Multispectral videos were collected with a telecentric lens attached to the nMSV apparatus and transillumination of the ear with a halogen 
light source, (a) Multispectral video frames comprised of the color camera image (Color), image of the NHA device (NIR), and unmixed detection bands 
(700-762 nm, 762-824 nm, 824-886 nm, and 886-948 nm) selected to reveal major changes due to a bolus injection of 2 mM ICG (See the 
supplementary video for ICG flowing through the piglet's ear), (b) Region of interest analysis of entire multispectral video showing the average pixel 
intensities in the region of interest defined by the blue and red dashed rectangle (inset) within the second detection band (762-824 nm) over time (blue 
and red). A PDD was used to monitor ICG concentration independently (green), (c) Magnitude Fourier transform of a time series where each time point 
represents the average pixel intensity in the 762-824 nm detection band for a given frame, (d) Maps for 700-762 nm and 762-824 nm spectral bands 
showing transmitted light intensities at the time of the maximum presence of ICG for each pixel in the respective detection band images, (e) Maps for time 
to maximum presence of ICG for pixels in 700-762 nm and 762-824 nm bands, (f) Histogram of time to maximum presence of ICG in 700-762 nm (top 
panel) and 762-824 nm (bottom panel), (g) Maps for 700-762 nm and 762-824 nm bands showing the correlation between the PDD signal and each 
pixel in the detection band images, (h) Relative correlation lag for pixels in bands 700-762 nm and 762-824 nm relative to the PDD signal, (i) Histogram 
of correlation lags between PDD signal and each pixel in 700-762 nm (top panel) and 762-824 nm (bottom panel) bands. 
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wavelength. Our interpretation is further supported by data 
presented in Figure 4 (f), which displays the histograms of the time 
to ICG peak measurements for the first two spectral bands. Analysis 
of the histograms revealed a 0.5 s time difference between the mean 
time to peak values of the first band and the second band. We 
hypothesized that the temporal shift between the two histograms 
was representative of measurements that were arterial-weighted in 
the first band and venous-weighted in the second band. In principle, 
the time to ICG peak extracted from each image pixel represented a 
temporal curve weighted by contributions from the arterial (early) 
and venous (late) vasculature. To test this hypothesis, maps of the 
correlation coefficient and correlation lag between the time-depend- 
ent PDD signal and the unmixed time series representative of each 
pixel were calculated (Figures 4 (g) and (h)). Similar to the time to 
ICG peak maps, the correlation coefficient maps exposed contrast 
features with low correlation values that were consistent with the 
superficial microvasculature observed in the color images. 
Furthermore, the maps contained diffuse areas of high correlation, 
which were likely representative of arteries and arterioles below the 
surface, since the PDD signal represents the arterial component. In 
addition, maps of the correlation lag (see Figure 4 (h)) and histo- 
grams of the correlation lag (see Figure 4 (i)) were similar to the maps 
(Figure 4 (e)) and histograms (Figure 4 (fj) extracted directly from 
the multispectral videos without prior knowledge of the PDD signal. 

Furthermore, we compared the temporal signals from early, mid- 
dle and late arrival of the time to ICG peak with the captured PDD 
signal from the hind foot of the piglet. In the analysis, motion arte- 
facts were reduced greatly by dividing data from the first band by data 
from the second band on a pixel-by-pixel basis. Figure 5 displays the 
average ratio of pixel intensity for the first band to the second band 
for image region areas where early, intermediate, and late time to 
ICG peak occurred. The PDD signal representative of the arterial 
component is shown for comparison. The signal from the region 
representative of early arrival had the highest correlation with the 
signal from the PDD. The early arrival signal had a narrower circula- 
tion bandwidth and had both dye recirculation trends in the signal. 
Therefore, the early arrival region was weighted toward the arterial 
component. The signal representative of the intermediate time to 
ICG peak region was broadened compared to the signal related to 
the early arrival region, but narrower than the region representative 
of the late arrival. Therefore, the signals related to regions of inter- 
mediate and late arrival of the ICG peak were weighted toward the 
venous component to a larger extent than in the early arrival region. 

Discussion 

It is evident from the piglet results that the nMSV apparatus provided 
a means to perform bolus tracking of an intravasculature dye within 
four spectral bands simultaneously. The high temporal sampling 
combined with simultaneous spectral measurements enabled the 
extraction of various temporal and spectral metrics attributed to 
the transit of dye, which permitted a distinction between regions 
based on relative weighting of the arterial and venous components. 
The high temporal sampling also enabled measurement of the heart 
rate. Moreover, spectral images facilitated the significant removal of 
motion artefacts to improve visualization of the bolus first-pass and 
the subtle dynamics of bolus recirculation. 

In future, the nMSV apparatus could be employed for localizing 
abnormal tissue during a surgical process when ICG is injected into 
the body (as described in Ref. 30). The captured images from the 
nMSV system could enhance detectability of the abnormal tissue as 
well as monitoring different physiological parameters such as blood 
flow and heart rate. Also, the nMSV apparatus can be used to mon- 
itor oxygenated and deoxygenated blood cells in vein and blood 
vessels. Therefore, one can speculate that the nMSV apparatus can 
be applied to the detection of cancerous skin lesions due to the 
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Figure 5 | Average signals from the first band (Bl) over the second band 
(B2) for early (red), intermediate (blue), and late (black) arrival of the 
ICG peak. The blue and black curves were offset relative to the red curve for 
clarity. The green curve represents the PDD signal. 

difference in oxygenated and deoxygenated blood concentrations 
in abnormal and normal skin lesions (as described in Ref. 31). 

We have presented a nanofilter-based multispectral videography 
system capable of unmixing a complex scene into images at 4 detec- 
tion bands within the range of 700 nm to 950 nm. The nMSV system 
utilized a tiled arrangement of identical blocks, where each block 
consisted of a mosaic of 4 SP-energy-matched NHAs. Each NHA 
in the block was fabricated with a unique periodicity selected to 
control its optical pass band. Multispectral videography was capable 
of recording the dispersal of droplets of dyes (MB + , CuS0 4 , and ICG) 
into water with specificity for the absorption properties for each dye. 
Furthermore, nMSV was able to capture the dynamic flow-induced 
changes in dye (ICG) concentration within a capillary tube. Finally, 
nMSV was used to track a bolus injection of dye (ICG) within the 
microvasculature of the ear of an anesthetized piglet. The multispec- 
tral videos were rich in features and video-based analysis provided 
physiological parameters such as average heart rate and venous- 
weighted flow dynamics of the microvasculature. Analysis of the 
multispectral videos in combination with supplemental PDD mea- 
surements resulted in estimates of the distribution of transit times 
through the microvasculature, and separation of the venous and 
arterial components. This technique holds promise for dynamic 
physiological monitoring of blood flow in tissues, and may have 
application to skin cancer detection and identification. 
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